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ABSTRACT 

Research on school effectiveness has shown that 
schools differ in regard to outcomes of comparable groups of 
students. Some schools are found to be more effective than others and 
these more effective schools have identifiable characteristics that 
help them excel. Outcomes measured to determine educational 
effectiveness include basic skills and knowledge, compensation or 
equity, social skills and attitudes, higher order skills, and several 
newer educational objectives. Educational effectiveness provides and 
requires a holistic theory on education, dealing with input, process, 
context, and products cf education. Other criteria are sometimes 
used, such as expenditures, to determine effectiveness. A useful 
theory about important factors in .the input of education must include 
factors like the student's and teacher's backgrounds and personal and 
financial resources for education. It is also important to 
distinguish between the significant factors at the class and school 
levels that contribute to educational effectiveness. Research on 
educational effectiveness needs to develop better designs for survey 
studies, better process measures, and better use of statistical 
techniques. (Contains 45 references.) (JPT) • 
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^ The concept of school effectiveness especially with respect to outcomes and processes is 
§ discussed. Based on the results of research, a model for educational effectiveness is 

developed which takes into account the factors at the classroom and school level and the 

relationship between the levels. For further elaboration of the model a relationship with 

the learning processes of students' should be constructed. 

1. Introduction 

Attention to educational effectiveness has its origins in research and practice with respect 
to school effectiveness. The early tesearch projects carried out by Brookover, Beady, 
Flood and Schweitzer (1979) and Edmonds (1979) in the USA and by Rutter, Maughan, 
Mortimore and Ouston (1979) in the UK have shown that schools differ from each other 
with respect to the outcomes of comparable groups of students. Some schools prove to be 
more effective than other schools and more effective schools have some characteristics in 
common which ineffective schools do not have. In educational policy making and practice 
the idea of effective schools also draws a great deal of attention, which is understandable 
because it offers possibilities of improving schools to get better results. In educational 
theory and research these results stimulated the start of research projects looking for 
factors that could explain effectiveness in education. Later on, serious criticism arose with 
respect to the methodology, the statistical analysis and the conceptual frameworks of the 
school effectiveness research (Purkey and Smith, 1983; Ralph and Fennessey, 1983; 
Reynolds, 1985).. 

Nevertheless, educational effectiveness is an important idea within the educational 
sciences. When the effectiveness of education at different levels of the system is analysed 
together, it can in a way be seen as the core matter of educational sciences and research. 
Educational research in this case is directed to explain the differences in educational 
outcomes based on a theory about causes and effects in education. In this sense 
6q educational effectiveness can be seen as a holistic theory about education which takes into 

^ account the outcomes of education, the inputs, the processes and the contexts in which 

\^ education takes place. 

In this sense it is a welcome addition to educational research in general. Such a 
programme has to address questions with respect to outcomes, criteria for effectiveness, 

N^v . , PERMISSION TO REPRODUCE THIS 

^\ inputs, processes and contexts. material has been granted by 
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2. Outcomes of education 

In the past, research on school effectiveness was criticised with respect to the criteria of 
effectiveness. Research took educational outcomes in the academic field as the only 
criterion and in addition the measurement of this criterion was quite poor involving, for 
example, the proportion of students going from primary to secondary education or marks 
in school exams. At this moment the best criterion for educational effectiveness is the 
value education adds to the initial attributes of students. Effectiveness is related to 
objectives in education which distinguishes educational effectiveness from the study of 
educational effects, which also takes into account unexpected outcomes of education like 
the results of the hidden curriculum on the one hand, and on the other hand is 
distinguished from the concept of educational efficiency which is concerned with the 
relationship between the effects of education and the inputs of education, most of the time 
in terms of finance. Added value conceptions stress the point that students have a 
background, an aptitude for learning, and a home environment and peer group, etc., 
which has already contributed to the knowledge skills students have at the moment 
education starts. With respect to educational effectiveness we have to take into account 
the student's background as well as the student's initial attributes with respect to the 
specific objectives under study. This requires measurement of abilities like intelligence 
and motivation as well as initial attributes with respect to the objectives under study, like 
performance in mathematics, reading, etc. 

Education stresses the point that the educational system contributes to educational 
outcomes at the different levels: the classroom level, the school level, and the contextual 
level. In research on educational effectiveness we have to specify the level under study 
and the factors at the various levels and in which way some levels can contribute to the 
processes at other levels. As said before, educational effectiveness restricts the criteria for 
effectiveness to what can be achieved by schools and what schools are for. School 
effectiveness research was criticised because it just takes into account superficial criteria, 
for example basic skills and knowledge. Therefore, at the moment within educational 
effectiveness multiple outcomes are proposed as criteria for effectiveness. The following 
outcomes are some of these multiple outcomes: 

Basic skills and knowledge , like reading, mathematics, language. Especially with 
respect to the background of educational effectiveness it is quite reasonable that in 
educational effectiveness there is such a great deal of attention given to basic skills 
and knowledge, because in these fields disadvantaged students did not succeed, 
(Brookover et. al, 1979; Edmonds, 1979) 

A criterion frequently used in the past for educational effectiveness was 
compensation for initial behaviour (equity). The idea of equity is connected with a 



belief in the school effectiveness movement that schools can compensate, more or 
less, for initial differences. Based on educational research so far with respect to 
effectiveness, one can conclude that the possibilities for schools to compensate for 
these differences are quite small (Brandsma, 1993; Van der Werf, Weide, and 
Tesser, 1991). These studies obtain almost the same result as the evaluation of so- 
called compensation programmes in the past, like Head-Start, Follow-Through and 
in the Netherlands the project Education and Social Environment, obtained 
(Slavenburg and Peters, 1989; Scheerens, 1987). Although equity is a longstanding 
aim in education, it turns out that schools do not contribute much to the reduction 
of differences. 

Social skills and attitudes , for example towards school and towards different 
school subjects. The idea behind this is that schools should be more than places 
for academic development and that schools should be directed to developing 
academic and cognitive skills but also social and aesthetic skills and, on top of 
that, to influencing attitudes that are important in their own respect, but which can 
also influence academic outcomes. 

Higher order skills , like problem solving, are, especially in higher grades, useful 
criteria for educational effectiveness. 

On top of that a broad range of 'new' educational objectives are formulated in 
different fields, like educational technology, creativity and moral behaviour. It 
became clear, next to the discussion about the aims of education, that schools 
differ in the way they can achieve different kinds of objectives (Mortimore, 
Sammons, Stoll, Lewis, and Ecob, 1988; Knuver, 1993). 

Some research results argue against the adoption of multiple outcomes within school 

effectiveness research immediately. 

1. One of the most striking findings of school effectiveness research in the earlier 
years was the fact that schools with a restricted set of outcomes which did not go 
for a broad range of educational objectives but restricted themselves to a small set 
of academic outcomes had better results than schools with a broad scope of 
educational outcomes. This holds especially for low SES schools (Teddlie and 
Stringfield, 1993). The recommendation for educational practice to steal time from 
other subjects for the basics can be seen as the practical implication of these 
empirical findings (Levine and Lezotte, 1990). 

2. In the evaluation of the innovation within primary education in the Netherlands it 
turned out that schools that were the most innovative with respect to educational 
goals (different goals than in the past) did not achieve very well. These schools 
that addressed themselves to new areas of schooling had the worst results not only 
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in the new areas of education but also in the so-called 'old areas', like reading, 
mathematics and language. In this case it means: the more you want the less you 
get. (Van der Werf, 1988) 

The suggestion that quality (or excellence) is something different than equity and 
that therefore these two are different objectives for education does not hold. An 
argument for this can be found in research carried out by Van der Werf and 
Weide (1991). In the evaluation of the Dutch educational priority programme, low 
effectiveness and high effectiveness schools were distinguished with respect to the 
results of Dutch as well as of immigrant students. These results are shown in 
Figure 1. It is clear from this figure that the variance in low effectiveness schools 
is quite high, that Dutch students in these schools perform uelow the general mean 
of Dutch students and that immigrant students in these schools perform far below 
the general mean of immigrant students. In high effectiveness schools immigrant 
students perform much better, although below the general mean of Dutch students, 
but they are close. Dutch students in these schools perform better than the general 
mean of Dutch students, although in this group the variance is smaller than in low 
effective schools. This means that 'going for quality' can also mean reducing the 
variance. This fact is contrary to results in most research, which shows that 
quality does not go along with reducing variance and even going for equity can 
increase variance, the so-called Matthew effect (Walberg, 1991). In the 
educational priority programme in the Netherlands a distinction is made between 
effective instruction and specific activities for immigrant students to reduce the gap 
between Dutch and immigrant students. In Table 1 activities related to effective 
instruction and to specific activities are summarised. Effective instruction 
characteristics are based on research in different areas, like direct instruction, 
research on grouping activities and research on student assessment. The specific 
activities are directed to improving language abilities and reducing the gap between 
the school and the home environment. 

Table 2 provides the results of the analysis between effective and ineffective 
schools with respect to the quality dimension, the equity dimension and the 
combination of quality and equity. Contrary to what one expects, there is no 
difference between effective instruction and specific activities with respect to the 
effects on quality and equity measures. In fact, most of the time activities related 
to effective instruction count for both high quality and equity. This means that 
quality and equity probably are not so different as suggested in the list of multiple 
outcomes. This is also contrary to the position of Nuttall et al. (1989), who 
suggest that effective schools are differentially effective for different groups. It 



seems that what is good for the brighter students in effective education could also 
be appropriate for disadvantaged students. 
4. In recent publications Knuver (1993) and Knuver and Brandsma (1993) also 
present the results of research on the relationship between so-called affective 
outcomes and academic outcomes in education. The background of the study was 
that in other studies it turned out that there is no relationship between academic 
and affective outcomes of education (Mortimore et aL, 1988) or even a negative 
relationship (Marsh, Smith, and Barnes, 1985). In the Dutch study it turns out that 
the academic outcomes have in fact an effect on the attitude towards arithmetic, 
towards schools and the well-being of students. This attitude has an effect on 
attitude in the next year as well as on the academic results of the following year. 
This indicates that attitudes, and other affective outcomes, are the result of 
academic outcomes. 

Based on the results of this study one should consider affective outcomes as the 
results of academic outcomes and one should not put too much emphasis on 
affective (and maybe also other outcomes) as separate independent results of 
education. 

The foregoing dealt with the criticisms with respect to outcomes, especially the criticism 
that the effect criterion in educational effectiveness research is not well chosen. There are 
more possibilities for criterion definition and from a technical point of view it is no 
problem to develop instruments, but based on research results we have to be careful about 
that. But this is just one aspect of criticism. Another point of criticism is that the effects 
of effective schools are insignificantly small. In fact, this is a more general problem about 
the influence of education as a whole. This has to do with the question as to what 
education contributes to the educational career of students. We know that the largest part 
of the variance in school results between students is explained by aptitude and SES, just a 
small proportion of variance can be explained by variables at the school and instructional 
level (see for instance Walberg, 1984). The proportion of variance that is left over after 
aptitude and SES is + 20%, depending on the study (and the statistical procedures). From 
these 20% just a small proportion is explained by factors wo have studied so far in school 
effectiveness research (less than 1 till, at most, 2-3%). But when differences between 
effective and non-effective schools are phrased in terms of their effects on the individual 
school careers of students, it turns out that these differences (even if they are quite small 
in the Netherlands) mean that there is a difference between referring or not referring to 
special education, or the retention in one grade or not and the choice of a higher level of 
secondary education. So, even when they are quite small in a statistical sense, they can be 
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very important for the individual careers of students in a policy sense (Bosker and 
Scheerens, 1989). 

There is a great deal of criticism with respect to the criteria of the objectives of 
education: it is also argued that the effects of schools are quite unstable. There are 
different figures for this in the different studies and the discrepancy between the results of 
the studies is quite large. Most of the time the correlation between school subjects within 
a grade is not that high in primary schools, which correlation is between .55 and .80 and 
in secondary schools between .45 and .75. This is also true between grades, in some 
studies this discrepancy is quite large between grade levels. In primary education 
correlations ?re found between .10 and .65 and in secondary education between .25 and 
.90 (Bosker, 1991). 

In summary: educational effectiveness provides and requires a holistic theory on 
education, dealing with input, process, context and products of education. The first 
question has to do with results, outcomes of education. This includes the criteria for 
educational effectiveness. From a technical point of view problems on the criterion side 
can be solved, but theoretically, on the conceptual side, there are some problems 
connected with the criteria themselves. 

In the past most of the educational effectiveness research used academic outcomes as the 
only criterion. This choice of criterion was criticised and at the time being pleas were 
made for multiple criteria for effectiveness. Even when we follow that line of thinking we 
will have to keep in mind that there are always arguments against use of these multiple 
outcomes. Also when the effects of effective schools on education are quite small, they 
can be very crucial with respect to the individual careers of students. A remaining 
problem is the instability of effectiveness, but this probably has not only to do with the 
choice of criterion but more with the instability of education as a whole, between 
subjects, between classes and between grades. 

3. Educational process: factors contributing to educational effectiveness 
In the foregoing section we dealt with the problem of criteria for effectiveness and most 
objectives of education can be treated that way. Sometimes other criteria are used also, 
like the amount of money involved to achieve objectives which in fact is an efficiency 
criterion and can be used additionally to effectiveness. But most of the time schools and 
classrooms are evaluated based on the extent to which they achieve educational objectives. 
But, in fact, a theory on educational effectiveness is not concerned in the first place with 
the criteria or the criteria alone, but more with the question of how these objectives can 
be achieved. Educational effectiveness deals with the question why schools with 
comparable pupils initially differ in the extent to which they achieve the objectives. What 
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are the processes, the factors that contribute to educational outcomes in effective schools? 
'Comparable' is a difficult concept. It can mean the same kind of children, the same 
intellectual and socio-economic background, but even in this case schools can differ in 
financial resources, the level of competence of teachers and the social context of the 
school. In a system approach a distinction is made between input, context and processes 
(next to product) of education. The input consists of all kinds of variables connected with 
financial or personal resources and with the background of students. By 'context' is meant 
the socio-economic and educational context of schools, for example the guidelines for 
education and the (national) evaluation systems. The most important factors concern the 
process which is going on at the classroom and school level. The question school 
effectiveness research deals with most of the time is what kind of factors within the 
school and classroom make a difference between effective and less effective schools. In 
fact, this question was the background of the school effectiveness movement that started 
with the first studies in this field by Brookover et al. (1979) and Edmonds (1979). Their 
research proved that schools differ in the extent to which they can achieve results with 
comparable groups of students. Early school effectiveness research was directed to finding 
the factors that made the distinction between effective and less effective schools. In these 
so-called 'outlier 5 studies evidence was found that a small number of factors contribute to 
effectiveness. Most famous in this case was the so-called five-factor model of Edmonds 
(1979). Later on this model was criticised from a methodological and conceptual point of 
view (Scheerens and Creemers, 1989). But in the early days of school effectiveness and 
school improvement the five-factor model and later on other models with some more 
factors in them, drew a great deal of attention from educational practice and policy 
making. It seemed quite easy to change schools from non-effective to effective by just 
introducing programmes in which some factors could be improved, like for example the 
evaluation of student progress in schools or (in-service) training for the improvement of 
the educational leadership of principals (Lezotte, 1989). 

Later on it turned out that it is not that easy to improve schools. Effective and non- 
effective schools differ in more than just a small amount of factors. This led to more 
research to distinguish between effective and non-effective schools. The earlier studies 
were mostly outlier studies, but, after criticism of the methodology of outlier studies, 
more survey studies were carried out, enlarging the list of characteristics of effective 
education. When the idea of effective education spread to other countries than the USA, 
replication studies were carried out to test whether or not the same characteristics of 
effective education could be found in other countries. The results of these studies did not 
confirm the list of factors produced by research in the USA. Generally speaking, one can 
conclude that on the one hand the list of characteristics was enlarged and that on the other 



hand the replication studies could not find much empirical evidence for these 'factors' or 
characteristics. In addition to this, a more conceptual approach was also advocated. Such 
a framework of theory could explain the differences between effective and non-effective 
education and this might be a point of departure for further research. 
In (replication) studies of the last years, the results of these three movements (outlier 
studies, survey studies and theoretical studies) are presented. Creemers and Lugthart 
(1989), Creemers and Knuver (1989), Creemers (1992a), Levine (1992), Levine and 
Lezotte (1990), Reynolds (1989, 1991, 1992), Scheerens (1990, 1992) and Stringfield 
and Schaffer (1991) sum up, each for their own country, factors that make a difference 
between effective and non-effective education within schools and classrooms. Especially 
interesting is the review of research provided by Levine and Lezotte in 1990. First they 
produce a list of factors as mentioned in Table 3, based on 400 studies on school 
effectiveness in the USA. This general list, which contains almost everything that can be 
found in schools and can be enlarged with 'further possible correlates', is broken down 
into other factors, for example, the correlates for effective instructional arrangement and 
implementation are given in Table 4. In total this results in a list of hundreds of correlates 
of effectiveness, more a result of research methods and techniques than the generation of 
genuine, clear and relevant concepts in a theory about effectiveness. 
In the correlational studies a large amount of schools and variables are involved. In this 
way, even small correlations can be significant. In the outlier studies mostly a few 
schools or classes but many factors are studied. Always some, or even a large number of, 
variables seem to distinguish between this small set of schools. Probably that is the reason 
why in replication studies quite a number of factors did not appear again. In Mortimore et 
al. (1989), a study in the UK, twelve factors could be found (Table 5 ). All of them are 
comparable with the factors mentioned by Levine and Lezotte, but Mortimore found less 
factors than Levine and Lezotte. Quite a number of the American factors did not prove to 
be very effective. In twelve Dutch studies even less factors could be found to distinguish 
between effective and non-effective schools, of which some provided evidence for the five 
factors distinguished by Edmonds. Scheerens and Creemers conclude that an orderly 
climate, frequent evaluation, achievement orientation, high expectations and direct 
instruction seem to contribute to effectiveness in the Netherlands (Scheerens and 
Creemers, 1989). 

Because of the criticism in the past, in recent years research has been improved, but 
research on educational effectiveness still has to deal with weaknesses with respect to: 



1. the conceptual framework; 

2. the limitation of effectiveness to a restricted set of components of education, the 
criterion problem and the outcome measures; 

3. the design of the research studies and instruments used for measurement of 
processes; 

4. the statistical procedures used so far. 

We dealt with the criterion problem in section 2, in the following we will concentrate on 
the development of the conceptual framework for educational effectiveness. Related to 
this we will touch on the methods and statistical procedures. An important contribution to 
educational effectiveness research is made by the development of a conceptual framework, 
because it can guide the search processes, the design of studies and later on the 
interpretation of the results. Based on studies carried out so far it is possible to make a 
list of promising factors of educational effectiveness that have to be taken into account by 
developing a theory. It became evident that 'time-on-task' and 'opportunity to learn' are 
important intermediate variables that can explain student outcomes. Instructional and 
school processes that could be important are the instructional arrangements, high 
expectations, orderly climate, educational leadership, a restricted set of objectives and a 
clear mission for the school, evaluation and monitoring and a positive school-home 
relationship (like involvement of parents and support from parents in homework). 
In recent years some models for school effectiveness have been developed. The basic idea 
behind all models for school effectiveness is to distinguish between more levels in 
education. At least all the models contain the individual student, learning, the classroom 
and the school level, and the higher levels in the model provide the conditions for wh^t 
happens at the levels below (see Figure 2). At each of these levels, factors contribute to 
the outcomes or form conditions for what happens at the lower levels. This means that 
not just one factor induces results but a combination of factors. 

Further developments of this basic model are provided by Stringfield and Slavin (1992), 
Scheerens (1991) and Creemers (1991). All these models have in common that they are 
more or less based on Carroll's model for school learning in which the time needed for 
mastering the educational objectives is a function of student characteristics - like ability 
and motivation - and the quality of teaching. Most of the models developed so far are 
quite precise at the instructional level. They make a distinction at that level between 
different components of instruction like the learning material, instructional behaviour and 
management behaviour of teachers and grouping procedures. The components like teacher 
behaviour can have characteristics that make a difference between effective and non- 
effective instruction. At the classroom level the characteristics of effective instruction of 
the components are related to time-on-task and opportunity to learn. The quality of 



material, grouping and behaviour is in one way or another related to the time available 
for learning (and teaching) and what happens within this time framework, to what content 
is offered related to the objectives (opportunity to learn). Except for grouping, the other 
components - material and behaviour - are also related to the adaptation of instruction to 
the level of students, through ensuring appropriateness, structuring of the arrangements 
including evaluation and monitoring, reinforcement and incentives. At the school level a 
distinction between factors and valvules is less clear. Based on ideas about how the 
school level can provide conditions for the instructional level and some insights from 
organisational theories, possible factors are discerned at the school level. In the QAIT- 
MACRO model (Stringfield and Slavin, 1992) these are meaningful goals, attention to 
academic functions, coordination, recruitment and training, and organisation. In the model 
provided by Scheerens (1990) the achievement orientation, the organisation of the school 
in terms of educational leadership and consensus, the quality of the school curricula in 
terms of content coverage, form and structure, and the orderly atmosphere are 
distinguished, showing again a combination of some curricular factors, objectives, content 
coverage, achievement orientation and some factors connected with the organisation of 
schools, like educational leadership. But how these factors can influence what goes on at 
the classroom level, between classes at the same grade level and between grade levels 
remains unclear. The same criticisms hold for the instructional model provided by 
Creemers (1991) in which again school curriculum variables and organisational variables 
are distinguished. Connected with this idea of some formal relationship between what 
goes on in classrooms, between classrooms and between the class and school level the 
ideas of consistency, cohesion, constancy and control are introduced. 
In the first (outlier) study within the International School Effectiveness Research Program 
(ISERP) the same bundles of variables will be investigated both at class and school level. 
In this way the relative strength of the variables at these two levels will be apparent, but 
also the class/school interface will be opened by studying the within-school, between-class 
variances in processes and outcomes in order to get information about consistency, 
cohesion, constancy and control. 

So far the models are not fully-fledged theories but just, as they call themselves, 
conceptual frameworks. They have to be developed based on further theory development 
and empirical research. The overall framework for further development can be sketched 
as in Figure 3. 

Some further explanation of the model 

In this model a distinction is made between achievement, educational attainment and 
output. The (ultimate) output can be the professional or educational career and the results 




on examinations, but the immediate output is the achievement on tests connected with 
educational objectives, the criterion for effectiveness. The students' background and their 
abilities, motivation, perseverance and aptitude, strongly determine their achievements. 
As stated before, other input factors are the resources of the school, teacher backgiound, 
experience and expectations. Time-on-task is in fact the time students are willing to spend 
on school learning, the educational task, but is also determined by processes at the school 
and instructional level. Time-on-task is the time students are really involved in learning 
and can be expanded by homework. But this time has to be filled by opportunities to 
learn. These opportunities concern the supply of learning material and experiences, 
exercises by which students can acquire knowledge and skills. In fact learning 
opportunities are the instructional operationalisation of the objectives of education, 
whereas tests are the evaluative operationalisation of the same objectives. In this respect 
one can speak about the content coverage of the curriculum. 

Based on meta-analysis of a number of studies it is possible to make a list of 
characteristics of the three components of classroom instruction, learning material, 
grouping procedures and teacher behaviour which influence effectiveness. 
With respect to learning material these are the following: 

the extent to which curricula offer opportunities to learn: quantity of subject matter 
offered, and degree of overlap between goals (that should be tested) and subject 
matter; 

explicitness and ordering of goals; 

structuring and clarity of subject matter (in relation with goals) ; 
use of advance organisers; 

the extent to which curricula evaluate student achievement. 
With respect to grouping procedures : 

mastery learning, heterogeneous grouping and cooperative learning can induce 

higher effectiveness; 

the effectiveness is dependent on: 

x availability of differentiated learning material 

x testing, feedback and corrective measures. 
With respect to teacher behaviour : 

management of the classroom; 

orderly and quiet atmosphere; 

high expectations; 

clear goal setting: 

x restricted set of objectives 

x emphasis on basic skills 
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x emphasis on cognitive learning and transfer; 

structuring the content: 

x sequence of objectives and content 

x advance organisers 

x making use of prior knowledge of students 

x immediate exercise after presentation of new content; 

questions (low order/higher order) wait time; 

evaluation/testing and feedback; 

corrective measures; 

pacing. 

Some of the above-mentioned characteristics have to be defined more precisely. For 
example with respect to learning material and teacher behaviour the term 'structuring' is 
used, but structuring does not mean anything without a further determination of what is 
meant by it. Structuring has to include giving attention to prior knowledge, the use of 
advance organisers, providing content according to the objectives in small steps, the clear 
presentation of central concepts, dividing the content into small units (including clarity in 
the presentation) and immediate exercises after presentation of the content. Structuring is 
not restricted to basic knowledge and skills but is also important in, for instance, 
scaffolding (Palincsar, 1989) in higher order learning. 

It is obvious that the teacher is the central component in instruction at the classroom 
level. He makes use of learning material and he actually carries out the grouping 
procedure in the classroom. But, on the other hand, the teacher needs learning material 
and in organising grouping procedures learning material that is consistent with the 
grouping procedure used is necessary. At the school level one can make a distinction 
between the educational arrangements of the school which includes the development plan 
of the school, and the organisation of the school, the structure and the processes going on 
there which can influence what happens at the classroom level and between classrooms. 
The educational policy of the school is codified, written down in the development plan 
and has to deal with the aims and objectives cf the school. In this respect a restricted set 
of objectives is important: structuring of the objectives in different grades, the transition 
between the grades, the evaluation policy, monitoring of students within grades and 
between grades, and the policy for adaptive instruction within the school with respect to 
the subjects and grades. The concept of the 'organisation of the school' covers the way 
schools try to secure this within grades, between classes and between grades, and is based 
on the notion that school policy with respect to education is carried out by teachers and 
students. School climate has to do with a quiet and orderly atmosphere, the responsibility 
teachers take for students' progress and the responsibilities students take for their own 
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learning. This relates to the educational leadership of the principal but also to the 
cohesion in the team and the control of students and teachers. In this respect the formal 
characteristics earlier mentioned are important, like: consistency in the classroom between 
textbook, teacher behaviour and grouping procedures; cohesion : every team member 
underlines the principles and behaves that way; constancy : during their whole school 
career (between grades) students receive the same 'treatment' and the control of students 
and teachers. 

Above the school level some contextual conditions which have to do with national policy 
can enhance school effectiveness, like programmes for educational assessment and the 
development of indicator systems, national guidelines for development plans or curricula . 
On the one hand they can explain differences between countries, but on the other hand it 
is far more important to look at the variation in how different schools deal with these 
national guidelines. 

4. Development of a theory on educational effectiveness 

In the foregoing sections we described for the whole educational system the relationship 
between the different levels and components, especially with respect to educational 
effectiveness. It turns out that outcomes of education have to be determined very 
precisely, not just because we need a criterion for effectiveness but also to guarantee that 
in educational effectiveness we take into account the important components of education. 
Based on that we can develop a theory about important factors in the input of education 
including factors like the student's and teacher's backgrounds and the personal and 
financial resources for education. What is more important is to make a distinction 
between the important factors at the class and school level with respect to education that 
contribute to educational effectiveness or, to put it differently, that can explain differences 
in outcomes between students in different classes and schools. 

It is clear that research programmes have to be carried out with respect to educational 
effectiveness. These research programmes are addressed to questions about effectiveness 
within and between countries to gain more insight into the contextual differences of 
educational effectiveness. For that reason we have to make better research designs for 
survey studies (ox even experimental studies) to develop appropriate instruments, like 
more outcome measures, better process measures at different levels of education and we 
have to use statistical techniques like multilevel techniques and LISREL analysis or even a 
combination of both. In addition to survey studies we need in-depth analyses and 
especially studies on schools in transition from effective to non-effective status or from a 
non-effective to an effective stage. 



Especially important at this moment is the further development of a conceptual framework 
even when this means a restriction of our focus to some components of the process of 
education in schools and classrooms. Most of the factors described are just a bunch of 
variables. The relationship between the above-mentioned models and the basic model (by 
Carroll) they are all related to is quite unclear. For that sake a relationship should be 
established between components at the instructional and school level and the components 
of Carroll's model. A vehicle for that development can be the main factors of 
effectiveness: 1) time for learning, 2) learning opportunities and 3) quality of education 
(at class, school and contextual level) (Creemers and Reynolds, 1993). 
Carroll's model about learning in schools can be criticised for the fact that it does not 
explain how learning itself takes place, but it provides an overview of important factors 
that contribute to learning outcomes. Next to student factors like motivation, attitude and 
perseverance, factors at the classroom level are included, like time/opportunity to learn 
and quality. In Carroll's model the factors at the school level are not included. Carroll's 
model itself is often used as a point of departure for development of theoretical 
frameworks for school effectiveness and educational effectiveness. 

The empirical support for factors at the school level was quite weak and there was a lack 
of theoretical linking between classroom and school factors. In the development of a 
middle range theory an attempt is made to develop a model for educational effectiveness 
that links all levels by defining the higher levels as conditional for the lower levels. 
Carroll's model is taken as a starting point. This means that, especially within the 
classroom and at the school level the concepts time, opportunity and quality are defined 
more precisely. 

4.1 Classroom level 

The original Carroll model does not pay much attention to the definition of factors at 
classroom level. Bloom (1976) elaborated some notions on quality of instruction in his 
theory on mastery learning, and the concept of content covered can be seen as a useful 
elaboration of the time allowed-factor. Slavin (1987) tried to build a model of effective 
instruction on the alterable elements of the Carroll model but the factors he mentions 
(quality of instruction, appropriateness of instruction, incentives and time use) do not 
reflect specific teacher behaviours any more than the original Carroll factors. Carroll 
(1989) emphasises the importance of going beyond the mere procedural aspects of 
instruction (such as frequency of testing) and looking for effective ways to present and 
organise contents of instruction. Information on these aspects can be found in teacher 
effectiveness process-product studies. 



14 J 5 



With respect to the process and input of education it became evident that academic 
learning time and time-on-task are closely related to the outcomes of education. 
According to the 'beginning teacher evaluation study' (Fischer et aL, 1981) and other 
studies later on, academic learning time can be defined in different ways as the time 
schedule, the amount of hours in the schedule devoted to subjects and ultimately the time 
students are engaged in learning. This means that at different levels of education time-on- 
task is an important issue and can be defined as a contributing factor to educational 
outcomes. Except for time itself, it is important what students learn, what they do in 
relation to the educational outcomes. Based on the results of the IEA studies one can 
conclude that the amount of time devoted to a specific subject within the time schedule of 
the class and the school is a good predictor for educational outcomes. The test taken at 
the end of the educational process can be seen as an evaluative operationalisation or 
definition of the objectives. Education and instruction as provided in classrooms and 
schools can be seen as a more educational or instructional operationalisation of the same 
objectives. 

Next to time and learning opportunities the quality of instruction and schooling is an 
important factor for educational effectiveness. By quality is meant those characteristics, 
factors, variables in instruction and the school as a whole that contribute to the 
explanation of differences in outcomes between students in different classes, schools and 
educational systems. According to Figure 3 the quality of instruction and the school 
influence the time and opportunity to learn. They enlarge the time for learning and 
'facilitate' the use of the opportunity to learn for the students. The characteristics of the 
quality of education at the classroom level are summed up in section 3. 

4.2 School level 

Looking at the well-known lists of effective school factors (Scheerens, 1992), it becomes 
clear that most of the factors (such as an orderly climate in the school, evaluating student 
achievement at school level) are in fact reflections of the indicators of quality of 
instruction and time allowed/opportunity to learn at classroom level. Because of a lack of 
research studies that analysed school and classroom level in one design, it is hard to say 
what the separate contribution of these factors might be in accounting for pupil level 
variance when controlling for classroom level factors, In any case, many school level 
factors are somewhat meaningless when they are not clearly linked to classroom factors 
(Creemers, 1992b). Even if they do have an independent effect on pupil achievement, it is 
still not clear how this effect comes about and how it should be interpreted. 
In the model the school level factors are defined as conditions for classroom level factors. 
This definition restricts the selection of school level factors to only those factors 
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conditional for and directly related to quality of instruction or to time allowed/opportunity 
to learn. 

According to Creemers (1991), school level factors should either promote cohesion 
between teachers (stimulate similar effective teacher behaviour in all classrooms) or 
control what is going on in classrooms. At the school level a distinction can be made 
between educational and organisational aspects. 

On the basis of these cohesion and control principles, the following indicators at school 
level can be described for quality of instruction with respect to the educational aspects: 
rules and agreements about all aspects of classroom instruction; 
an evaluation system at school level to check pupil achievement, to prevent 
learning problems or to correct problems at an early stage (regular testing, 
remedial teaching, student counselling, homework assistance). 
With respect to the organisational aspects of the school level important factors are: 

a school policy on intervision and supervision of teachers, section leaders and 
school principals (educational leadership); 

a school policy to correct and further professionalise teachers who do not live up 

to the school standards. 
Indicators of time at the school level are: 

the development and provision of a time schedule for subjects and topics; 

rules and agreements about time use, including the school policy on homework, 

pupil absenteeism, cancellation of lessons; 

the maintaining of order in the school. 
Indicators of opportunity to learn at the school level are: 

development and availability of a curriculum, school working plan or activity plan; 

rules and agreements about how to proceed, how to follow the curriculum, 

especially with respect to transition from one class to another or from one grade to 

another. 

Creemers (1991) points at the importance of continuity in all indicators mentioned above, 
meaning that schools should not change rules and policies every other year. This 
constancy principle, however, can only be found in a longitudinal setting, by comparing 
school level factor* from year to year, and for this reason is not included in the model. 

4,3 Context, macro level 

The same components as mentioned before, quality, time and opportunity to learn can be 
distinguished on a national level. Quality regards the availability of an indicator system or 
national policy on evaluation. Time refers to the national guidelines with respect to the 
time schedule for schools and opportunity to learn refers to the national guidelines and 
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rules with respect to the topics/subjects to be treated in schools, such as a national 
curriculum. The already mentioned evaluation system has a control mechanism in it for 
time and opportunity to learn. It is clear that at the different levels also resources are 
important, but resources can be defined in the way it is done in this model: availability of 
material, teachers and other components supporting education in schools and classrooms. 
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(Van der Werf and Weicle, 1991, p. 239) 
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Table 1 



Instructional characteristics 



Effective instruction Specific activities 



Differentiation 

(grouping) 

(adaptive instruction) 
Whole group instruction 
Minimal competency levels 
Orderly climate 

Rehearsing 

Repetition 
Evaluation 

Registration of progress 

Academic learning time 
Homework 

Importance of cognitive objectives 
Orientation on content 



Correction of language 



Separate language lessons 
(language) 

Specific textbooks 
(language) 



Extra learning 

(language) 

Special material 
(language) 

Promotion of reading activities 

Activities for immigrant parents 



(Van der Werf & Weide, 1991, p. 235) 
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Table 2 

Differences between teachers 





mean 




scale 




high 
quality 


low 
quality 




Whole group education 


5.5 


4.1 


2-10 


Orderly climate 


39.6 


36.1 


9-45 


Correction of language 


2.6 


9.9 


3-15 


Separate language lessons 


12.6 


2.8 


1-5 


Activities for immigrant 
parents 


8.1 


9.9 


4-12 




high equity 


low equity 




Whole group education 


5.4 


4.1 


2-10 


Orderly climate 


40.3 


37.9 


9-45 


opeciiic memoes 


z.y 


1.5 


1-5 


Orientation on content 


24.9 


21.5 


7-35 




high q/c 


low q/c 




Orientation on content 


26.6 


21.6 


7-35 


Importance of cognitive 


40.6 


28.0 


1-100 



objectives 

(Van der Werf and Weide, 1991, p. 240) 
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Table 3 



Productive school climate and culture 

Focus on student acquisition of central learning skills 

Appropriate monitoring of student progress 

Practice-oriented staff development at the school site 

Outstanding leadership 

Salient parent involvement 

Effective instructional arrangement and implementation 

High operationalized expectations and requirements for students 

Other possible correlates 

(Levine and Lezotte, 1990, p. 10) 
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Successful grouping and related 
organizational arrangements 

Appropriate pacing and alignment 

Active/enriched learning 

Effective teaching practices 

Emphasis on higher order learning in 
assessing instructional outcomes 

Coordination in curriculum and instruction 

Easy availability of abundant, appropriate 
instructional materials 

Classroom adaptation 

Stealing time for reading, 
language and math 
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Table 5 



Purposeful leadership 

The involvement of the deputy-head 

The involvement of teachers 

Consistency among teachers 

Structured lessons 

Intellectually challenging teaching 

Work-centered environment 

Limited focus within sessions 

Maximum communication between 
teachers and pupils 

Record-keeping 

Parental involvement 

Positive climate 



(Mortimore, Sammons, Stoll, Lewis and Ecob, 1989) 
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A model of school effectiveness 



School level 
Inputs, structure 
managerial processes 



Context 



Classroom level 
Inputs, teaching 
strategies 



Background variables 
SES, aptitude 



Individual student level 
Achievement 
educ. attainment 



(Scheerens and Creemers, 1989) 
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Conceptual Framework for Educa tional Effectiveness 



Educational 
effectiveness 



National curricular 
guidelines 
Indicator system 
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School 
effectiveness 



School curriculum 
'Mission 1 organisation 
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Monitoring system 
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Policy 



Process 



Instructional 
effectiveness 



Grouping procedure 
Method/learning material 
Teacher (instructional 
management) 
Behaviour 



Input 



Time on task 
Opportunity to 
learn 
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of students " 
(SES) 



Achievement 

Educational 

attainment 



Motivation 
"Aptitude 



Output 



(Creemers, 1991) 
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